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ABSTRACT 

Results of the Behavior Assessment System for 
Children (BASC) Student Observation Scale (SOS), a measure of 
classroom behavior, were correlated with results of the BASC Teacher 
Rating Scale (TRS) . Two classroom observations were made of each of 
30 students (21 males and 9 females) aged 5 to 11 years. Teachers of 
those students completed the TRS. Correlations were computed by 
comparing scales on the TRS with the SOS scales. Selected TRS items 
were also compared to SOS scales. The stability of the SOS was 
determined by correlating the two observations with each other. 
Eleven of the 14 TRS subscales were validated by low to moderate 
correlatiors with SOS subscales. Furthermore, all of the selected TRS 
items correlated significantly with the SOS Adaptive and Maladaptive 
Totals. These results provide a stringent test of the validity of the 
TRS. Three tables present study data, and 36 references are included. 
(Author/SLD) 



Reproductions supplied by EDRS are the best that can be made 
* from the original document » * 

^ y jV yr ^ Vr ^ Vr Vf >V Vr A Vr Vr ^ V( V( Vr tV Vt Vr ^ ^ ^ yt y : iV Vr Vr ^ A Vt * yf -^f 5fr Vf Vr ^ Vr /: 



BASC TRS 
1 



in 

o 

CO 
Q 



\j* OIPAHTMENT or EDUCATION 

fOUQATlONAL RESOURCES tNfOWMATlON 
CENTER (ERICI 



'ifiS <Jocum«nJ h«s tJ«*n f«produted 

CI MinOf c^^ang** Men "^•rt* lo iTtOrove 
teprodiictton Quilitr 

• Point* o' **w o« c>o<f»ions >l»l»din ihifdocu 
rT<»n) (Jo not ntcesMnly tepi-Ment o"<c.i«' 

OfRl position O' DOliT/ 



■PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTE-H (ERlCl 



Validation cf the BASC Teacher Rating Scale 
by the BASC Student Observation Scale 



Nancy J. Lett and Randy W. Kamphaus 



University of Georgia 



^ Running head: VALIDATION OF THE BASC TRS 

^ BEST COPY ftmABLE 



BASC TRS 
2 



Abstract 

In a study of the criterion related validity of the BASC 
Teacher Rating Scale (TRS) , results of the BASC Student 
Observation Scale (SOS) , a measure of classroom behavior, v.ere 
correlated with results of the TRS. Two classroom observations 
of each of 30 students ages 6 to 11 were performed. Teachers of 
those students completed the TRS- Correlations were computed 
comparing scales on the TRS with the SOS scales. Selected TRS 
items were also compared to SOS scales. .:t'>oility of the SOS va 
determined by correlating the two observations with each other. 
Eleven of the fourteen TRS sabscales were valic-c.'ced by low to 
moderate correlations with SOS subscales. Furchei r.ore, all of 
the selected TRS items correlated significantly with the SOS 
Adaptive and Maladaptive Totals. These results provide a 
stringent test of the validity of the TRS, 
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Validation of the BASC Teacher Rating Scale by the BASC Student 

Observation Scale 

Objectively measuring children's behavior in the classroom 
has consistently been a problem in school psychology. Behavior 
varies across settings, tasks ^ and caregivers, and psychologists 
will observe few behavior problems during testing in the clinic 
or school setting (Barkley, 1987; Barkley, 1990; Glutting & 
McDermott, 1988) . The most popular nethod of behavioral 
assessment is currently the behavior rating scale to be completed 
by parents or teachers. Rating scales are an integral part of 
the diagncsis of Attention ^eficit Hyperactivity Disorder, 
Conduct Disorder, Behavior Disorder, and Oppositional Defiant 
iriscrder azicng others. P>2.tir:cj scales 3.re tirr ^"■e'^f icient .■ 
inexpensive, ?nd require little specialized training on the part 
of the rater (Martin, 1988) (Mayes, 1987). Data can be gathered 
on infrequently occurring behaviors which may not be seen during 
a classroom observation, and raters with much experience with the 
child can supply important information (Barkley, 1990) . Rating 
scales with excellent psychometric properties are available 
(Darl;ley, 1987; Barkley, 1990) . 

There are, however, some problems with this type of measure. 
Rater bias involving over-reporting or under-reporting behaviors, 
halo effects, rater misunderstanding of questions on the scale, 
and dependency on the accurau.** memory of the rater are some of 
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the disadvantages of the behavior rating scale (Barkley, 1987; 
Mayer., 1987; Platzman, Stoy, Brown, Coles, Smith, and Falek, in 
press) . Other problems include subjectivity, differences in 
tolerance for particular behaviors, differences in raters' 
experience with chilaren, and rater bias (Mayes, 1987; Madle, 
Neisworth, & Kurtz, 1930; Klein & Gittelman-Klein, 1974; Klein & 
Gittelman-Klein, 1975) . Some studies have found a decrease in 
reporting of behavior problems over time when th^ same child is 
rated more than once by the same rater (Neisworth, Kurtz, Jones, 
St Madle, 1974 ; Glow, Glow, & Rump, 1982 ; Milieu, Roberts, Loney, 
& Caputo, 1980) . This finding may be the result of regression to 
the mean (Milich et al., 1980). 

oGcausa of probie.T»s inherent in the use cf behavior rating 
scales, other objective measures should be used to validate and 
SUT^ dement th.es?. r'B suits. A cls.ss'^ccn observation "'itilizi'^cr 
momentary time-sampling is one method of validation. Advantages 
of direct observation include observation of behavior in the 
natural environment (Barkley, 1987) , objectivity, the use of 
operationally defined behavioral categories, little or no 
regression to the mean, and less likelihood of rater bias (Mayes, 
1987). Studies of various direct observation systems have 
demonstrated high interobserver reliabilities. Percentage of 
agreement between scorers is usually above 80% and is typically 
greater than 90% (Tindal & Parker, 1987; ?Iorn, Conners, Wells, & 
Shaw, 1986; Schworra and Birnbaum, 1989; Bailey, Bender, & 
Montgomery, 1983; Rapport, DuPaul, Stoner & Jones, 1986; Eaton, 
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Enns & Presse, '"987; Steinkamp, 1980; Rapport, Jones, DuPaul , 
Kelly, Gardner, Tucker & Shea, 1987) . 

There are, however, disadvantages to the use of classirooin 
observation. It is expensive and time consuming. Furthermore, 
because behavior varies over situations, the observation may not 
be representative of the child's typical activities, (Abikoff, 
Gittelman-Klein & Klein, 1977; Mayes, 1987). classroom 
observation techniques may not be sensitive to events such as 
aggression which occur infrequently but are highly salient 
(Atkins & Pelham, 1991) . In addition, reactivity or observer 
effects may be a problem. The presence of observers in the 
classroom may elicit atypical behavior on tne part of some 
children (Barkley, 1990), This phenc-encn may be r.cre of a 
problem with hyperactive children. In a study by Abikoff et al 
(1977) teachers indicated that during classroorr. cbservat ions , 
43.7% of the hyperactive children exhibited behavior that was nor 
representative of their usual behavior, reas only 13.8% of the 
comparison group exhibited atypical behavior. However, Abikoff 
et al. (1977) ODserved each child on five different occasions and 
found no significant differences between the first observations 
and the last. Dubey, Kent, O'Leary, Broderick, and O'Leary 
(1977) also found no evidence of observer effects during 
classroom observations. 

Normative data is not available for classroom observation 
scales, and if collected, it may not be valid across different 
classroom situations (Barkley, 1987; Barkley, 1990; Mayes, 1987; 
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Zentali, 1980) • Remedies for this problem have been suggested by 
Barkley (1987; 1990). Collection of local normative data or use 
of yoked controls during each observation may "provide an 
indication of deviance." 

The purpose of t.he current study was to assess the validity 
of a new teacher rating scale by correlating the results of the 
Student Observation Scale, a component of the Behavior Assessment 
System for Children (BASC) (Reynolds & Kamphaus, in press) , with 
the BASC Teacher Rating Scale. Historically, the correlation 
between teacher rating scales and objective measures has been in 
the low to moderate range (Kazdin, Esveldt-Dawson & Loar, 1983; 
Blunden, Spring & Greenberg, 1974; Barkley & Ullman, 1975; 
Victor, Halverson, Inoff u Buczkowski, 1972; Atkins, Pelhar, & 
Licht, 1985; Milich & Fitzgerald, 1985; Brown, 1982; Horn, et 
al., 193 6; 3a i ley et al , , 19 82; Eaton et al., 19 87), Variables 
such as subjectivity and halo effects on rating scales, and for 
direct observation, insensitivity to qualitative differences in 
behavior (Abikoff & Gittelman, 1985) , variability of behavior 
over settings, l. d insensitivity to low rate, highly salient 
behaviors may be some of the reasons for low to moderate 
correlations » However, important clinical information jan result 
from divergent data (Barkley, 1990; Gordon, DiNiro, Mettelman & 
Tallmadge, 1989) . Most researchers point out the importance of 
collectinr both teacher ratings and classroom observations during 
child assessment (Barkley, 1990; Campbell, 1985; Douglas, 1990). 
Knowing the amount of correlation to expect betisreen these 
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measures and the reasons for possible disparity will assist the 
clinician in assessment and diagnosis. 

Specifically, this study assessed the following questic is. 

1. Do the subscales of the BASC Teacher Rating Scale (TRS) 
exhibit a low to moderate correlation with the subscales of the 
Student Observation Scale (SOS) as would be predicted from 
previous research? 

2. Are some subscales of the TRS more highly correlated 
with the SOS scores than others? 

3. Do the Adaptive and Maladaptive totals of the BASC SOS 
exhibit low to moderate correlations with the TRS subscales? 

4. Do selected items from the BASC TRS correlate with SOS 
Adaptive, Maladaptive, and selected subscale scores? 

5. Does the BASC SOS show adequate stability over a one to 
eioht '^ee}^ oeriod? 

Method 

Subjects . Thirty children between the ages of 5 and 11 who 
had been referred for psychoeducational assessment as a result of 
school problems or as a re-evaluation of their special education 
placement were the subjects of this study. The children were 
from several counties in Northeast Georgia and had been referred 
to a university-based clinic or to their local school district 
for evaluation. Referral questions included Attention Deficit 
Hyperactivity Disorder, Learning Disability, Behavior Disorder, 
and Mild Mental Retardation. 
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The sample consisted of 21 boys and 9 girls who ranged in 
age from 5 to 11 with a mean age of 8« With regard to race, 20 
of the children were white and 10 were African-- American. 
Intelligence scores ranged froni 62 to 118 with a mean of 88 
(sd=14) . 

I nstruments . The Student Observation Scale (SOS) is part of 
the BASC (Reynolds & Kamphaus, 1992), a system of behavioral 
assessment which includes a number of components. In addition to 
the Student Observation Scale, the BASC includes Teacher Rating 
Scales, Parent Rating Scales, a Parent Personality Profile, a 
Self-report of Personality, and a Structured Developmental 
History (parent interview) (Reynolds & Kamphaus, 1992)* 

Tht^ Student: Observarion Scale urilizes a 15 minuTie morientary 
time-sampling technique* Every thirty seconds during the 
momentary time'^sampling component, tile Qjbser^'er vatches tris 
child's behavior for 3 seconds. At the end of the 3 seconds, the 
behaviors are recorded by placing check marks in the appropriate 
behavioral categories. Space is provided for comments on specific 
behaviors observed. Twenty-seven seconds are allowed for 
recording of behaviors. This is a relatively long period of time 
and allows even novice observers to complete the observation 
easily and accurately. Categories were developed empirically by 
asking teachers and school psychologists to list behaviors seen 
in the classroom and by factor analyzing results of observations. 
The SOS categories include Responding to Teacher/Lesson, Peer 
Interaction (appropriate), Working on School Subjects, Transition 
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Movement (appropriate) , Disruptive Movement, Inappropriate 
Vocalization, Somatization, Repetitive Motor Movements, 
Aggression, Self-Injurious Behavior, Inappropriate Sexual 
Behavior, and Bowel/Bladder Problems. These categories fall into 
two domains. Adaptive Behavior and Maladaptive Behavior. 

The BASC Teacher Rating Scale (Reynolds & Kamphaus, 1992) 
for children ages 6 through 11 was also utilized. The Teacher 
Rating Scale consists of 148 questions about children's behavior 
to which the teacher responds "never," "sometimes," "often," or 
"almost always." Subscales derived from the Teacher Rating Scale 
are Hyperactive/Impulsive, Aggression, Conduct Problems, Anxiety, 
Depression, Withdrawal, Somatization, Attention Problems, 
Learning Problems, Atypical Psychotic, Adaptability, Social 
Skills, Leadership, and Study Skills. The BASC manual (Reynolds 
^ K^TTi'ohaus , 1992) reports ^ concurrent v-iliditv st— dv ccrreliEitin'u 
the BASC TRS with the Burks' Behavior Rating Scales. Validity 
coefficients for similar scales ranged from .81 to .86. For 
example, BASC TRS Anxiety correlated .85 with the Burks' 
Excessive Anxiety; BASC TRS Attention Problems correlated .86 
with Burks' Poor Attention; and the correlation between BASC TRS 
Hyperactive/Impulsive and Burks' Poor Impulse Control was ,84. 

Observers « Seven graduate students in a School Psychology 
program were the observers. The SOS is designed to be used by 
observers with a limited amount of training, therefore, the 
graduate students were trained in the use of the student 
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Observation Scale by one of the authors in a 30 minute training 
session* 

Proced ure , Each child was observed in his or her classroon 
during a structured period of the day, and behaviors were 
recorded on the BASC Student Observation Scale. Two observations 
were made for each child. The second ooservation was completed 
within 8 weeks of the first in order to investigate whether or 
not differences between scores on the SOS and the TRS were due to 
problems with stability of the SOS. A copy of the BASC Teacher 
Rating Scale was left with the teacher who completed the form and 
mailed it to the authors of the study. 

Data analysis . Pearson correlation coefficientr were used 
as estimates of the relationship between TRS and SOS subscales 
and items, Intraclass correlation coefficients were used to 
assess test-retest reliability of the SOS, 
Results 

T RS Subscales correlated with SOS Adaptive Sc M aladaptive 
Totals. Correlation coefficients comparing the first 15 minute 
SOS Adaptive Behavior Totals and the Maladaptive Behavior Totals 
with each TRS subscale showed low to moderate correlations 
between seven of the comparisons (See Table 1) . The TRS Conduct 
Problems subscale correlated ,66 with the Maladaptive SOS Total 
and -.59 with the Adaptive SOS Total. Similarly, TRS Aggression 
and SOS Maladaptive Behavior showed a .52 correlation. TRS 
Depression correlated .46 with SOS Maladaptive Behavior and -.42 
with Adaptive Behavior, 

1 ^ 
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A uniq\ie feature of the BASC TRS is its inclusion of 4 
adaptive behavior subscales: Adaptability, Social Skills, 
Leadership, and Study skills. The results of data analysis for 
this study showed a .55 correlation between the TRS Adaptability 
subscale and the SOS Adaptive Behavior Total. Small sample size 
may have been the reason that correlations between SOS Adaptive 
Behavior and other TRS Adaptive subscales did not reach 
significance. The correlation between TRS Adaptability and SOS 
Maladaptive Behavior was -.58. 



Insert Table 1 about here 



TPS Subscales C orrelated with SOS Subscales . Table 1 also 
shows correlations betiweeii TRS and SOS subscales, Nip.t^tieen of 
the correlations were statistically significant (alpha <.05), 
The SOS Disruptive Movement subscale produced the largest 
correlations with TRS subscales. SOS Disruptive Movement 
correlated .66 with TRS Aggression, .53 with TRS Conduct 
Problems/ and .44 with TRS Impulsivity , Additionally, Disruptive 
Movement showed a -.55 correlation with TRS Adaptability. 

SOS Inappropriate Vocalization correlated .50 with TRS 
Aggression and .53 with TRS Conduct Problems. An examination of 
the TRS Aggression subscale items shows that many of the items 
relate to verbal aggression: Argues when denied own way; 
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Threatens to hurt others; Blames otheiTs; Talks back to teachers; 
Orders others around; :alls other children names; etc. 

Surprisingly, SOS Inattention did not show a significant 
correlation with TRS Attention Problems, SOS Inattention did, 
however, correlate .48 with TRS Withdrawal, and .43 with Learning 
Problems. SOS Inattention correlated negatively with the TRS 
Adaptive Subscales: -.4^ with Social Skills, -*49 with 
Leadership, and -.55 with Study Skills. 

T RS Items Correlated With S0> : . TRS items that had clear 
counterparts on the SOS were analyzed separately to assess their 
criterion related validity. Results of TRS items such as: Reads 
assigned chapters; Is eas ly distracted from class v. rk; Talks 
too loud; and Doesn't pay attention, were correlated with SOS 
totals .^nd with selected subscales. All of the selected TRS 
iter.s correlated significantly with the first 15 inir.ute SOS 
Adaptive and Maladaptive Totals. Correlations from -.38 to -.55 
were found between 8 TRS items describing problem behaviors and 
the SOS Adaptive Behavior Totals. Correlations between the 
selected items and the SOS Maladaptive Behavior Composite ranged 
from .33 to .49 and are presented in Table 2. 



Insert Table 2 about here 
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Test-Retest Reliability . Test-retest reliability over a one 
to eight week period was calculared for the SOS Adaptive and 
Maladaptive Behavior scores by comparing the first 15 ininute 
observation with the second 15 minute observation. The 
reliability coefficient for the SOS Adaptive Behavior score was 
.44, and the reliability for the SOS Maladaptive Behavior score 
was .46. 

Intraclass correlation coefficients were computed coinparing 
the individual SOS subscales for each 15 minute observation. 
Disruptive Movement (r=.62). Inappropriate Vocalization (r^.43), 
and Repetitive Motor Movement (r==.75) exhibited adequate test- 
retest reliability over an eight week period (see Table 3) . 



Insert Table 3 abou^ here 



Di scussion 

The current study lends some support to the validity of the 
BASC Teacher Rating Scale (TRS) for children ages 6-11. Eleven 
of the fourteen TRS subscales were validated by low to moderate 
correlations with subscales from the BASC Student Observation 
Scale which utilizes a 15 minute momentary time sampling 
procedure during a classroom observation. Among the best 
validated TRS subscales were: TRS Hyperactivity which was 
supported by its correlation with SOS Disruptive Movement? TRS 
Aggression which correlated with SOS Disruptive Movement and 
Inappropriate Vocalization; TRS Conduct Problems which exhibited 
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a negative correlation v/ith SOS Responds to Teacher and positive 
correlations with Disruptive Movement and Inappropriate 
Vocalization; and TRS Adapatabil ity which was supported by a 
positive correlation with the SOS Adaptability total and ^ 
negative correlation with the SOS Maladaptive total. In general, 
the TRS problem behaviors correlated negatively with the SOS 
Adaptive total and positively the SOS Maladaptive total, 
similarly, the TRS adaptive subscales were supported by positive 
correlations with the SOS Adaptive total. 

Surprisirgly , the SOS was not able the validate the TRS 
Attention Problems scale. This may be related the the fact that 
SOS Inattention scale exhibited low test-retest reliability. 
Inax:i:en;:ion may be difficult zo observe objectively in the 
classroom. The early form of the BASC SOS that was used in this 
study defined inattsnt^.'^n cis "dii'^-^dire^i^ ng" ^r-*^ Qv^cQ;>~i^ro>~ i^^,*""^^'^^,^ ~ 
indicated that many of them scored this category when the child 
was looking around or staring into space. The current version of 
the BASC SOS defines Inattention as "staring blankly/daydreaming; 
doodling; looking around; looking at hands? and fiddling with 
objects/ fingers." Hopefully, this definition will aid observer:, 
in recording this category more reliably - 

Internalizing scales of the TRS were the most poorly 
validated. Teachers, however, may not be so sensitive to these 
types of behaviors. Similarly, observers may not be able to 
"observe'* the cognitions associated with these problems. 
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Another problem with the results were the test-retest 
reliability data. Although we did not expect extrem<^ly high 
correlations due to the fact that observations were collected 
during different classroom activities from one observation to the 
next, we were rather disappointed by correlations in the 40 's for 
the Adaptive and Maladaptive totals. The second observations 
were performed by first year graduate students who were trained 
in the use of the SOS, but who seemed to have more difficulty in 
correctly coding behavior than the advanced graduate students who 
performed the first observation. Their inexperience with 
standardized r.c?.sting procedures may have hurt the accuracy of 
their data collection. Another study of the test-retest 
reliability of the SOS is recommended. 

In general, however, these results support the validity of 

S'^wC "^^S* a certa'^ri degree, ws c^a*^ s^*'^ "'pd^'C^.t'o^s t'*^*"-' 

behaviors that teachers report on teacher rating scales when we 
go into the classroom for an observation. Of course it goes 
without saying that the use of classroom observations augment the 
information provided by teachers in rating scales by giving us a 
first-hand look at behavior in the classroom environment. Rating 
scale problems with reporter bias, halo effects, and dependence 
on the memory of the rater are overcome with classroom 
observation using momentary time-sampling. Direct observations 
will likely never replace teacher rating scales, and the results 
of the current study indicate that a differential diagnosis is 
not clearly indicated by results of the SOS. However, the SOS 
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Table 1 

Correlation of TFS with SOS Subscales and Totals 



Student Observation Scale (SOS) TOTALS 





Respon 


Peer 


Work 










Repet. 








to 


Inter- 


on 


Trans. Disrupt, 




inapp. 


Motor 


Adap. 


MaL 


TRS 


Teacher action 


School Move. 


Move. 


Inattention 


Vocal. 


Me/a 


Total 


Total 


Hyperactivity 


-.04 


.44* 


-.07 


,19 


.44' 


-.23 


.22 


-.08 


.05 


.20 


Agression 


-.28 


.24 


-.03 


-.04 


.66* • 


-.11 


,60" 


-.14 


-.35 


.52" 


Conduct. Prot). 


-.37* 


.26 


-.14 


-.01 


.53** 


,11 


.51" 


.16 


-.59* 


.66" 


Anxiety 


-.37' 


.03 


.24 


-.20 


.21 


.03 


.33 


.16 


-.20 


.34 


Depression 


-.43' 


.07 


.09 


-.09 


.27 


.18 


.46* 


.03 


-.42* 


.46' 


Withdrawal 


.05 


-.19 


-.14 


-.26 


-.02 


.48" 


-,02 


.21 


-33 


.34 


Somatization 


-.03 


-.16 


.01 


-.24 


-.08 


.30 


-.24 


.28 


-.18 


.15 


Attention Prob. 


.24 


.16 


-.22 


05 


-.01 


.16 


-.22 


.08 


-,03 


.04 


Laarning Prob. 


.18 


-.07 


-.07 


-.20 


-.43* 


AS* 


"29 


.26 


- 01 


-.03 


Atypical Psy. 


.16 


.23 


-.29 


-.04 


,23 


.08 


.15 


.10 


-.21 


.29 


Adaptability 


.11 


-.21 


.38' 


-.15 


-.55- • 


-.30 


-.29 


.09 


.55* 


-.58" 


Social Skills 


-.14 


-.13 


.47* 


-.25 


-.07 


-.44* 


.03 


.19 


,36 


-.19 


Leadership 


-.15 


.24 


.29 


.03 


.14 


-.49" 


.22 


.22 


.31 


,00 


Study Skills 


-.11 


.05 


.22 


-.01 


.20 


-.55" 


.38 


.03 


.20 


- 04 



Note- • alpha <05 
" alpha <01 



Table 2 

Correlation of Selected TRS Items with SOS Total Scores 

Adaptive Maladaptive 
Total Total 



Reads assignt.d chapters. 


-.38' 


.45* 


Is easily distracted from ciasswork. 


-.55* 


.40' 


Talks too loud. 


-.49* 


.44' 


Doesn*t pay attention. 


-.51" 


.40' 


Taps foot or pencil 


-.56*' 


.40' 


Has short attention span. 


-.47" 


.38* 


Calls out in class. 


-.49* 


.48'* 


Interrupts others wtien speaking. 


-.54' • 


.49*' 


Listens attentively. 


-.48'* 


.34 


Listens to directions. 


-,47"' 


.34 



Note- * alpha <05 
** alpha <.0l 



Table 3 

bsca*Q Test-Retes t Reliability 

Intractass Correlation Coeflicients 



Respond to Teacher -.06 

Peer Interaction .28 

Work On School .10 

Transition Movement 26 

Disruptive Movement -76 

Inattention .25 

inappropriate Vocalization .59 

Repetitive Motor Movement .74 



